1. Field of the Invention
The present invention relates to a vehicle environment monitoring device which identifies objects in the vehicle environment by performing binarization processing of images taken by infrared cameras.
2. Description of Related Art
Conventional vehicle environment monitoring devices identify objects with which the vehicle has a chance of colliding, such as pedestrians, within images of the environment of the vehicle taken by an imaging device such as an infrared camera, and provides this information to the driver of the vehicle. In these apparatuses, parts of the images taken of the vehicle environment by a left and right pair of stereo cameras which show high temperatures are assigned as objects, the distance to the objects is calculated by determining the parallax thereof, and from the direction of movement and position of the objects, those which are likely to affect the traveling of the vehicle are detected, and a warning is output (see Japanese Unexamined Patent Application, First Publication No. 2001-6096, for example).
However, if the identification of pedestrians is performed based only on shape determination as in conventional devices, then due to factors such as the pedestrian wearing a hat, the effect of clothing, or the environment around the pedestrian, the binary shape of the pedestrians themselves on the infrared image may be indeterminate. Moreover, when a vehicle is traveling in general, variation in the shape of the road surface ahead of the vehicle or pitching of the vehicle have an effect on the image, with the result that the heights of a pedestrian, whether an adult or child, may not be detected accurately. Consequently, there is a possibility that the on screen coordinates of the center of gravity of the object cannot be fixed with respect to distance, and the presence of pedestrians alone cannot be determined in a stable manner. Here, in addition to calculating the size of an object in real space from a gray scale image as in conventional devices, methods have been proposed in which only pedestrian-like objects are extracted based on the positional relationships of binarized objects, or in which road side structures and vehicles are extracted from the binarized objects, declared as non-pedestrian objects, and eliminated from consideration as objects which justify a warning.
Furthermore, heat reservoirs such as billboards, walls and utility poles in the general environment, which do not emit heat themselves but store heat received from external sources, tend to disappear from the infrared image (are no longer detected by the infrared cameras) due to their temperature dropping during rainfall, and with heat generators such as vending machines which themselves generate heat, although these are detected by the infrared cameras, during rainfall the portion of infrared radiation reduces (to almost zero). Hence, it is difficult to determine a shape accurately in rainfall conditions. In the same manner, because rainfall can cause variation in the radiant quantity of infrared rays, although exposed parts of people (the head and the like) will be detected, parts covered with clothing will not be detected by the infrared cameras because the clothing is wet. In this manner, the state of the vehicle environment can be different even at the same location depending on whether it is raining or not, and there is variation in the shape of all of the objects on the grayscale images detected by the infrared cameras. Therefore, conventional methods are proposed in which rainfall is detected based on signals from the operation of the wipers of the vehicle or a detection signal of a raindrop sensor, and separate processing is used in normal weather than is used during rainfall.
In addition, methods are proposed in which the state of a histogram of the entire image is used to determine whether or not rain is falling.
Under weather conditions where a large amount of rain continues to fall, it is possible to detect pedestrians in a stable manner by detecting rainfall based on signals from the operation of the wipers of the vehicle or detected signals of a raindrop sensor, and performing separate processing for normal times and times of rainfall, but in different conditions, such as when drizzle or rain begins to fall, ceases falling, or has just finished falling, it is possible that the identification of pedestrians only in a stable manner is impaired. Specifically, because there is no link between the rainfall determination processing performed using the signals from the operation of the wipers of the vehicle or detected signals of a raindrop sensor, and the pedestrian determination processing which identifies pedestrians within the infrared images, under conditions such as when drizzle or rain begins to fall, ceases falling, or has just finished falling, if the signals from the operation of the wipers of the vehicle or detected signals of a raindrop sensor call for the processing to be performed as for normal conditions despite the fact that the pedestrian determination processing for wet weather conditions is more appropriate, it is possible that the ability to determine the shapes of wet bodies accurately is impaired.
Furthermore, the use of a histogram of the entire image to determine whether or not rain is falling is effective in cases where, in an image of a body and its background, the histogram shows variation between times of normal weather and rain in both the body and its background. However, there is a problem in that in cases where there is no variation in the histogram in the background between times of normal weather and rain (if the background of the body is a view of nature such as a rice field, for example), the accuracy of the rainfall determination drops. Specifically, in a situation where no histogram variation is apparent in the background between normal weather and rain and there is one body, it is possible that even if the histogram of the body changes, the amount of variation relative to the histogram of the entire image is too small, which means that variation cannot be distinguished in the histogram in terms of the overall image, and rain cannot be detected accurately (the state of the body cannot be determined).